Method and apparatus for providing a multi-screen based multi-dimension game service

ABSTRACT

A method and apparatus provides a multi-screen based multi-dimensional game service using a home network and a home server. The method includes determining a resolution of a game screen to be served based on terminal information provided from a user terminal, storing a view on a user location in a multi-dimensional shared game space as a game session, generating an object for the view based on the determined resolution, performing multi-view rendering using multi-dimensional scene data and a user game input, capturing video and audio generated through the multi-view rendering, encoding the captured video and audio, and transmitting encoded video and audio to the user terminal.

RELATED APPLICATIONS(S)

This application claims the benefit of Korean Patent Application No.10-2012-0031109, filed on Mar. 27, 2012 and Korean Patent ApplicationNo. 10-2012-0072271, filed on Jul. 3, 2012, which are herebyincorporated by references as if fully set forth herein.

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for providing amulti-screen based multi-dimensional game service using a home serverand a home network, and more particularly to a method and apparatus forproviding a multi-screen based multi-dimensional game service, which issuitable for rendering and streaming scenes with a resolutionappropriate for a user terminal based on game inputs of a user and userterminal information when a game is continuously executed by a pluralityof user terminals or when a plurality of users plays a game.

BACKGROUND OF THE INVENTION

Recently, as a technology for a network and various user terminals isdeveloped, a demand for a multi-screen service is rapidly increasing. Auser can receive a service for a single content by moving one userterminal to another user terminal through the multi-screen service.

In particular, a computer game service is being developed into anon-line game or server-end type game service in which a user can play agame with other users through a home network or an internet network,escaping from a conventional game service in which a user plays a gameindependently in a personal computer (PC). Such a game service in thehome is being provided with a further advanced function through agame-dedicated console. The game-dedicated console can recognizemovement of a user using a stereo camera in a home network.

However, a multi-screen service based game service, which has beenprovided on a home network using the existing game-dedicated console, isnot provided yet at a user terminal.

In addition, an approaching method for the conventional multi-screenservice transcodes one media according to a resolution of a userterminal and provides the transcoded media. As a result, both decodingand encoding should be performed on an original media.

Herein, the transcoding is a process of converting a media having acertain resolution and media property into a media having a differentresolution and media property. The transcoding includes a decodingprocess for an original media and an encoding process for a decodedmedia to generate a target media to be converted. The transcodingrequires various processing methods and speeds according to the originalmedia to be converted, so that it is difficult to satisfy a responsetime of a game service.

Therefore, in a game service in which a plurality of users plays a gamein a single shared space, it takes a long time to process a mediaaccording to properties of a user terminal, such as a resolution. One offundamental reasons causing this problem is obtained by applying aconventional scheme that renders a game in the same manner andtranscodes the rendered game without obtaining resolution information ofthe user terminal in advance.

When serving moving picture contents of a movie using the conventionalmulti-screen service, one original media should be transcoded accordingto user terminal information and then provided to a user. Since,however, a game is a real-time media whose scenes change according togame inputs of a user, game scenes are newly rendered according to thegame inputs of the user and then provided to the user. Thus, it isdifferent from a case of using a media, such as movie contents, which isprepared in advance. Traditionally, this game scene rendering isperformed using a game engine or a game application software. Therefore,it is not effective to use the conventional scheme to provide a media toa small number of terminals existing in a home network, in aspects of aprocessing time and computing resource utilization.

Therefore, in case of playing a three-dimensional shared game throughterminals in a home network, a multi-view rendering is more effective toreduce a processing time and to improve computing resource utilization.In the multi-view rendering, after information on shared scenes isloaded on a server in advance, the server receives game inputs and viewinformation of each user are processes the received information.

There is a need to combine a multi-screen service using a home serverwith user terminal information and game input information. Meanwhile,recently, a technology of processing various types of software servicesin a server is being activated on the strength of the development of acloud computing related system and service technology. As a result,there is a demand of introducing a service scheme in which a home serverprocesses encoding/streaming and rendering of game scenes, which requirea lot of arithmetic operations, using the server-end computingtechnology, and the user terminal in the home network decodes anddisplays the game scenes.

SUMMARY OF THE INVENTION

It is, therefore, an object of the present invention to solve a problemsuch as a difficulty in providing a multi-screen service according to achange of a user terminal when a multi-dimensional (e.g.,three-dimensional) shared game is served in a conventional servicerelated to a multi-screen service using a home network.

In accordance with an embodiment of the present invention, an inputsignal of a game user is input through a home network, and a userterminal information management and multi-view rendering are performedin a home server. Therefore, it is possible to provide a scheme capableof providing a game on the home network even if a user terminal does notinclude high specification hardware.

In addition, in accordance with an embodiment of the present invention,a processing speed can be improved with encoding only even if thetranscoding for a multi-screen service is not preformed. Therefore, itis possible to provide a new scheme capable of transferring a service byminimizing a delay time according to the game playing of a user.

In accordance with an aspect of the present invention, there is provideda method for providing a multi-screen based multi-dimensional gameservice, the method including: determining a resolution of a game screento be served based on user terminal information provided from a userterminal; storing a view for a user location on a multi-dimensionalshared game space as a game session; generating an object for the viewbased on the determined resolution, and performing multi-view renderingusing multi-dimensional scene data and game inputs of a user; capturingvideo and audio generated through the multi-view rendering; and encodingthe captured video and audio, and transmitting the encoded video andaudio to the user terminal.

In accordance with another aspect of the present invention, there isprovided an apparatus for providing a multi-screen basedmulti-dimensional game service, the apparatus including: a user terminalmanagement block configured to determine a resolution of a game screento be served based on user terminal information provided from a userterminal; a user game session management block configured to store aview for a user location on a multi-dimensional shared game space as agame session; a multi-view rendering execution block configured togenerate an object for the view based on the determined resolution, andperform multi-view rendering using multi-dimensional scene data and gameinputs of a user; an encoding block configured to encode video and audiogenerated through the multi-view rendering; and an encoding scene streamprocessing block configured to process the encoded video and audio togenerate a game moving picture stream, and transmit the game movingpicture stream to the user terminal.

In accordance with still another aspect of the present invention, thereis provided a method for providing a multi-screen basedmulti-dimensional game service, the method including: determining aresolution of a game screen to be served based on user terminalinformation provided from a user terminal; checking whether or not agame session for the user terminal exists; if the game session exists,obtaining a view from the game session and adding the user view onto amulti-dimensional shared game space; if the game session does not exist,storing a view for an initial location of a user on themulti-dimensional shared gate space as the game session; generating anobject for the view obtained from the game session or the view for theinitial location based on the determined resolution, and performingmulti-view rendering using multi-dimensional scene data and game inputsof the user; capturing video and audio generated through the multi-viewrendering; and encoding the captured video and audio, and transmittingthe encoded video and audio to the user terminal.

Embodiments of the present invention are directed to a method andapparatus for providing a multi-screen based multi-dimensional gameservice, which determines a resolution of a game screen to be servedbased on terminal information provided from a user terminal, stores aview on a user location in a multi-dimensional shared game space as agame session, generates an object for the view based on the determinedresolution, performs multi-view rendering using multi-dimensional scenedata and a user game input, captures video and audio generated throughthe multi-view rendering, encodes the captured video and audio, andtransmits encoded video and audio to the user terminal. Therefore, it ispossible to provide a game on a home network even if the user terminaldoes not include high specification hardware. In addition, by improvinga processing speed with encoding only even though the transcoding for amulti-screen service is not preformed, it is possible to minimize adelay time according to the game playing of a user and reduce a gameresponse time of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects and features of the present invention willbecome apparent from the following description of embodiments given inconjunction with the accompanying drawings, in which:

FIG. 1 illustrates a schematic diagram of a game service system suitablefor applying a method for providing a multi-screen basedmulti-dimensional game service in accordance with an embodiment of thepresent invention;

FIG. 2 illustrates a block diagram of an apparatus for providing amulti-screen based multi-dimensional game service in accordance with anembodiment of the present invention;

FIG. 3 is a flowchart illustrating processes of providing a multi-screenbased multi-dimensional game service in accordance with an embodiment ofthe present invention;

FIG. 4 is a flowchart illustrating processes of providing a multi-screenbased multi-dimensional game service in accordance with an embodiment ofthe present invention;

FIG. 5 illustrates a result of performing a game rendering when a userterminal is a desktop, according to an embodiment of the presentinvention; and

FIG. 6 illustrates a result of performing a multi-view game renderingwhen a plurality of user terminals uses both a desktop and a smartphone, according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Unlike a conventional scheme of transcoding and encoding game scenesaccording to a resolution of a user terminal, embodiments of the presentinvention are directed to a scheme of determining a resolution of a gamescreen to be served based on terminal information provided from a userterminal, storing a view on a user location in a multi-dimensionalshared game space as a game session, generating an object for the viewbased on the determined resolution, performing multi-view renderingusing multi-dimensional scene data and a user game input, capturingvideo and audio generated through the multi-view rendering, encoding thecaptured video and audio, and transmitting encoded video and audio tothe user terminal. Therefore, it is possible to effectively improvedrawbacks of the conventional scheme using these technical means.

In the following description of the present invention, if the detaileddescription of the already known structure and operation may confuse thesubject matter of the present invention, the detailed descriptionthereof will be omitted. The following terms are terminologies definedby considering functions in the embodiments of the present invention andmay be changed operators intend for the invention and practice. Hence,the terms should be defined throughout the description of the presentinvention.

Hereinafter, embodiments of the present invention will be described indetail with reference to the accompanying drawings so that they can bereadily implemented by those skilled in the art.

FIG. 1 illustrates a schematic diagram of a game service system suitablefor applying a method for providing a multi-screen basedmulti-dimensional game service in accordance with an embodiment of thepresent invention.

Referring to FIG. 1, the game service system includes a home server 110connected to a home network 120, and a user terminal 130 such as adesktop personal computer (PC), a notebook, a smart pad, a smart phone,or the like.

Herein, a game is not played using a central processing unit (CPU) and agraphic processing unit (GPU) of the user terminal 130. Instead, astreaming based game service scheme is used. The streaming based gameservice scheme performs game service components such as game animation,sound generation, artificial intelligence system functions, physicalsimulation, rendering, and so on using hardware resources, such as a CPUand a GPU, which exist in the home server 110 connected to the homenetwork 120, and streams images showing game results to a user.

In particular, the home server 110 shares multi-dimensional game scenedata using views of a plurality of users for a multi-dimensional sharedgame (e.g., a three-dimensional shared game, a four-dimensional sharedgame, etc.), renders the multi-dimensional game scene data by applying aresolution of the user terminal 130 and a view of each user, and streamsthe rendered data. In accordance with an embodiment of the presentinvention, in the home, the home server 110 performs game rendering andencoding using resolutions of various user terminals and a view of theuser in the multi-dimensional game, and transmits a result of the gamerendering and encoding to the user terminal 130. That is, the homeserver 110 connected to the home network 120 can perform functions ofstreaming, encoding, and multi-dimensional game rendering for themulti-screen service.

The user terminal 130 processes game inputs of the user for themulti-screen based shared game service. That is, the user terminal 130transmits the game inputs to the home server 110. In addition, the userterminal 130 decodes result media (video/audio) received from the homeserver 110 through the home network 120, and then outputs/displays thedecoded result media through a screen, a speaker, and so on. If acertain user receives a service on a game progress as moving from oneterminal to another terminal, the home server 110 provides a seamlessmulti-screen service by applying view information stored in a usersession and user terminal information (resolution).

The user terminal 130 receives a game service from the home server 110through the wired or wireless home network 120 according to its type. Atthis time, the user terminal 130 can receive a streaming based gameservice with a resolution suitable for the user terminal 130. In theuser terminal 130, the game inputs of the user may be performed througha keyboard, a mouse, a touch input, and so on according to a property ofthe user terminal 130. The home server 110 processes the game inputsbased on an input mapping table pre-stored therein using an input logicof a served game, the game inputs, and information on the user terminal130.

FIG. 2 illustrates a block diagram of an apparatus for providing amulti-screen based multi-dimensional game service in accordance with anembodiment of the present invention. The apparatus includes amulti-screen processing block 202, a user terminal management block 204,a user game session management block 206, a user input processing block208, a multi-view rendering execution block 210, a game scene captureblock 212, an encoding block 214, and an encoding scene streamprocessing block 216.

Referring to FIG. 2, the multi-screen processing block 202 starts amulti-dimensional shared game service when it receives a game servicestart request from the user terminal 130 in the home, which is connectedto the home network 120. The multi-screen processing block 202 transmitsuser terminal information (e.g., resolution) to the user terminalmanagement block 204 and transfers view information on a location of auser on a multi-dimensional (e.g., three-dimensional, four-dimensional,and so on) shared game space to the user game session management block206. In addition, the multi-screen processing block 202 transmits gameinputs of the user, which are received while the game service is inprogress, to the user input processing block 208.

Herein, the user location on the multi-dimensional space may include alocation of an observer, which is a three-dimensional view, i.e., Xp,Yp, and Zp, a focal point where the observer looks at, i.e., Xf, Yf, andZf, and a field of view (FOV).

The user terminal management block 204 determines a resolution of a gamescreen to be served based on the user terminal information received fromthe user terminal 130. For instance, if the user terminal 130 is thedesktop PC, the resolution to be served may be determined as 1600×1200.If the user terminal 130 is the smart phone, the resolution may bedetermined as 640×480. Information on the determined resolution isprovided to the multi-view rendering execution block 210.

The user game session management block 206 stores a current view (gameview) with respect to the user location on the multi-dimensional sharedgame space as a game session in an internal memory (not shown). If agame session is terminated after the user played a three-dimensionalshared game in a previous time through the user terminal 130 and pausedthe game, the user view information on a three-dimensional space isstored in the user game session management block 206. The user viewstored as the game session may be used when the user resumes the sharedgame service later.

Herein, the user location on the three-dimensional space may include alocation of an observer, which is a three-dimensional view, i.e., Xp,Yp, and Zp, a focal point where the observer looks at, i.e., Xf, Yf, andZf, and a field of view (FOV).

The user game session management block 206 searches the internal memoryto check whether or not the game session for the user terminal 130exists in the internal memory when there is a shared game servicerequest from the user. If the game session exists in the internalmemory, the user game session management block 206 obtains the user viewstored in the internal memory and then adds the obtained user view tothe multi-dimensional shared game space. On the other hand, if the gamesession does not exist in the internal memory, the user game sessionmanagement block 206 stores a view for the user location on themulti-dimensional shared game space as a game session.

The user input processing block 208 processes the game inputs of theuser who is performing the multi-dimensional shared game service, andtransmits the processed game inputs to the multi-view renderingexecution block 210.

The multi-view rendering execution block 210 generates an object for thegame view received from the user game session management bock 206 basedon the resolution determined by the user terminal management block 204,and performs multi-view rendering using the game inputs of the user andmulti-dimensional scene data. In case of an initial execution of asingle user, the multi-view rendering execution block 210 generates anobject for an initial view, performs the multi-view rendering using gameinputs of the user and three-dimensional scene data that is previouslyprepared, and generates one frame of game progress images. Herein, themulti-view rendering may be performed using a rendering API (ApplicationProgramming Interface), such as OpenGL, DirectX, or the like, or a gameengine well known in the art. One rendered image is stored in a framebuffer region.

The game scene capture block 212 captures a scene result rendered byusers, i.e., one rendered scene image (video) and game sound (audio),from a frame buffer (not shown), and transfers the captured video andaudio to the encoding block 214.

The encoding block 214 combines the captured scene image (video) andgame sound (audio) transferred from the game scene capture block 212,and encodes the combined video and audio. The encoding block 214 mayperform the encoding using an MPEG-4 encoder or an H.264 encoder. Thatis, the encoding block 214 performs the encoding by combining video andaudio based on a frame number of a scene in a game, and transfersencoded data to the encoding scene stream processing block 216.

The encoding scene stream processing block 216 processes the encodeddata provided thereto to generate a multi-dimensional game movingpicture stream, and transfers the game moving picture stream to the userterminal 130. The encoding scene stream processing block 216 maytransmit the game moving picture stream to the user terminal 130 using areal-time protocol (RTP) or a real-time streaming protocol (RTSP).

Hereinafter, a sequence of processes for providing a multi-screen basedmulti-dimensional game service to users will be described using theapparatus that has the above-described configuration.

FIG. 3 is a flowchart illustrating processes of providing a multi-screenbased multi-dimensional game service in accordance with an embodiment ofthe present invention.

Referring to FIG. 3, if the user terminal 130 accesses the home server110 and requests a multi-dimensional shared game service, the userterminal management block 204 collects and determines a resolution to beserved based on user terminal information provided from the userterminal 130 at step 302.

After that, the user game session management block 206 stores or adds aview for an initial location of a user on a multi-dimensional sharedgame space as a game session in an internal memory at step 304. At thistime, the user input processing block 208 processes game inputs of theuser for the multi-dimensional shared game and transfers the processedgame inputs to the multi-view rendering execution block 210.

The multi-view rendering execution block 210 receives the game inputsand the view of the user, and renders the multi-dimensional game sceneusing a rendering API, such as OpenGL, DirectX, or the like, or a gameengine well known in the art at step 306. One rendered image is storedin a frame buffer region.

After that, the game scene capture block 212 captures one rendered sceneimage (video) and game sound (audio) from the frame buffer, andtransfers the captured video and audio to the encoding block 214 at step308.

As a result, the encoding block 214 combines and encodes the capturedscene image and game sound using an MPEG-4 encoder or an H.264 encoder.The encoding scene stream processing block 216 processes the encodeddata to generate a multi-dimensional game moving picture stream, andtransmits/streams the multi-dimensional game moving picture stream tothe user terminal 130 through the home network 120 at step 310. Herein,the game moving picture stream may be transmitted using a real-timeprotocol (RTP) or a real-time streaming protocol (RTSP).

Thereafter, the user terminal 130 decodes the video and audio data forthe rendered result transmitted thereto using, e.g., an MPEG-4 decoderor an H.264 decoder, and then outputs the decoded result through ascreen and a speaker. That is, the decoded video is displayed throughthe screen, and the decoded audio is reproduced through an audioregenerator. These processes performed in the user terminal 130 arerepeatedly performed while the user plays the multi-dimensional sharedgame.

FIG. 4 is a flowchart illustrating processes of providing a multi-screenbased multi-dimensional game service in accordance with an embodiment ofthe present invention.

First of all, the multi-dimensional game service may include a case thata user previously executes a game through the user terminal 130 at thehome server 110 and then pauses the game to terminate a game session. Aprocedure for processing this is required.

Referring to FIG. 4, if the user terminal 130 accesses the home server110 and requests a multi-dimensional shared game service, the userterminal management block 204 collects and determines a resolution to beserved based on user terminal information transferred from the userterminal 130 at step 402.

After that, the user game session management block 206 searches theinternal memory and checks whether or not a game session for the userterminal 130 exists in the internal memory at step 404.

As a result of the checking, if it is determined that there exists thegame session, a user view stored in the internal memory is obtained andadded onto a multi-dimensional shared game space at step 406.

As a result of the checking at step 404, if it is determined that thegame session does not exist in the internal memory, a current view for auser location on the multi-dimensional shared game space is added/storedas a game session at step 408.

After that, rendering for the user view and game inputs, renderingresult capture, encoding and streaming, and so on are performed at steps410 to 414. The processes at steps 410 to 414 are substantially the sameas those at steps 306 to 310 of FIG. 3. Therefore, for the simplicity ofthe description, the detailed explanation for steps 410 to 414 isomitted.

FIG. 5 illustrates a result of performing game rendering according to anembodiment of the present invention, when a user terminal is a desktop.

FIG. 5 shows a screen obtained by setting a service resolution to1600×1200, rendering a game scene, and transmitting the rendered resultto the user terminal 130, in the case that the user terminal 130 is thedesktop PC.

FIG. 6 illustrates a result of performing multi-view game renderingaccording to an embodiment of the present invention, when a plurality ofuser terminals includes a desktop and a smart phone.

FIG. 6 shows a screen obtained by setting a service resolution for thedesktop PC and the smart phone to 1600×1200 and 640×480, respectively,rendering a game scene, and transmitting the rendered result to the userterminal 130, in the case that the user terminal 130 includes thedesktop PC and the smart phone.

While the invention has been shown and described with respect to thepreferred embodiments, the present invention is not limited thereto. Itwill be understood by those skilled in the art that various changes andmodifications may be made without departing from the scope of theinvention as defined in the following claims.

What is claimed is:
 1. A method for providing a multi-screen basedmulti-dimensional game service, the method comprising: determining aresolution of a game screen to be served based on user terminalinformation provided from a user terminal; storing a view for a userlocation on a multi-dimensional shared game space as a game session;generating an object for the view based on the determined resolution,and performing multi-view rendering using multi-dimensional scene dataand game inputs of a user; capturing video and audio generated throughthe multi-view rendering; and encoding the captured video and audio, andtransmitting the encoded video and audio to the user terminal.
 2. Themethod of claim 1, wherein the view comprises a location of an observer,a focal point where the observer looks at, and a field of view.
 3. Themethod of claim 1, wherein the encoding is performed using an MPEG-4encoder or an H.264 encoder.
 4. The method of claim 1, wherein theencoded video and audio are transmitted using a real-time protocol (RTP)or a real-time streaming protocol (RTSP).
 5. An apparatus for providinga multi-screen based multi-dimensional game service, the apparatuscomprising: a user terminal management block configured to determine aresolution of a game screen to be served based on user terminalinformation provided from a user terminal; a user game sessionmanagement block configured to store a view for a user location on amulti-dimensional shared game space as a game session; a multi-viewrendering execution block configured to generate an object for the viewbased on the determined resolution, and perform multi-view renderingusing multi-dimensional scene data and game inputs of a user; anencoding block configured to encode video and audio generated throughthe multi-view rendering; and an encoding scene stream processing blockconfigured to process the encoded video and audio to generate a gamemoving picture stream, and transmit the game moving picture stream tothe user terminal.
 6. The apparatus of claim 5, wherein the viewcomprises a location of an observer, a focal point where the observerlooks at, and a field of view.
 7. The apparatus of claim 5, wherein theencoding block comprises an MPEG-4 encoder or an H.264 encoder.
 8. Theapparatus of claim 5, wherein the encoding scene stream processing blockis configured to transmit the game moving picture stream using areal-time protocol (RTP) or a real-time streaming protocol (RTSP).
 9. Amethod for providing a multi-screen based multi-dimensional gameservice, the method comprising: determining a resolution of a gamescreen to be served based on user terminal information provided from auser terminal; checking whether or not a game session for the userterminal exists; if the game session exists, obtaining a view from thegame session and adding the user view onto a multi-dimensional sharedgame space; if the game session does not exist, storing a view for aninitial location of a user on the multi-dimensional shared gate space asthe game session; generating an object for the view obtained from thegame session or the view for the initial location based on thedetermined resolution, and performing multi-view rendering usingmulti-dimensional scene data and game inputs of the user; capturingvideo and audio generated through the multi-view rendering; and encodingthe captured video and audio, and transmitting the encoded video andaudio to the user terminal.
 10. The method of claim 9, wherein a viewcomprises a location of an observer, a focal point where the observerlooks at, and a field of view.
 11. The method of claim 9, wherein theencoding is performed using an MPEG-4 encoder or an H.264 encoder. 12.The method of claim 9, wherein the encoded video and audio aretransmitted using a real-time protocol (RTP) or a real-time streamingprotocol (RTSP).